Many modern automobiles include integrated digital camera systems. Typical applications for these digital camera systems are blind spot monitoring, lane departure warning, rear view blind spot view, collision avoidance, traffic sign recognition, and drowsy driver detection. Typically, these systems incorporate visible or near infrared (NIR) capable image sensors.
In many cases, the digital image data captured by the digital cameras is never rendered or viewed, but rather used in a machine vision system as information for making decisions. Once the digital image data has been analyzed and a decision has been made, the digital image data is generally discarded. The hardware and software required for the image processing, analysis and subsequent decisions and information transmission is designed and optimized for the specific application, and is often integrated and co-located with the camera.
In other cases, the digital image data may be displayed on a soft-copy display for viewing by an occupant of the automobile. For example, rear view or “back-up” cameras, are used to display live video to the driver to aid in safely maneuvering and avoiding otherwise unseen objects while moving in reverse. However, as with the machine vision applications, the digital images are generally not stored once they have been displayed.
These digital camera systems are separately integrated as modules in vehicles, and are specialized, optimized and dedicated solely to the specific application of that particular system, such as one of the examples cited above. FIG. 1 shows a top view diagram of a prior art motor vehicle imaging system 100 including a motor vehicle 105 fitted with four digital camera imaging systems: a blind-spot imaging system including two blind-spot cameras 110, one in each side view mirror; a forward-looking imaging system including a forward-looking camera 111 mounted on the front of the vehicle; an interior imaging system including an interior-view camera 112 mounted to view the driver and occupants of the vehicle; and a rear-view imaging system including a rear-view camera 113 mounted at the rear of the motor vehicle 105. All four of the separately integrated digital camera systems do not interact or interconnect and are dedicated to perform specific functions.
The blind-spot imaging system is used for the applications of blind spot monitoring, blind spot warning, and lane change assistance. The blind-spot imaging system captures digital images using the blind-spot cameras 110 to detect objects in the blind spot of the operator of the motor vehicle 105. The captured digital images are analyzed by a signal processor integrated into and dedicated to the blind-spot imaging system to provide warning information to the operator. The digital images are not displayed or rendered for viewing, and are discarded after analysis.
The forward looking imaging system is typically used for applications such as night vision, adaptive cruise control, collision avoidance, lane departure warning, and pedestrian detection. Digital images captured using the forward-looking camera 111 are analyzed by a signal processor integrated into and dedicated to the forward looking imaging system in order to make decisions for the described applications. Typically, they are not rendered for view or display, and are subsequently deleted.
The interior imaging system is typically used for applications such as drowsy driver detection, air bag deployment assistance, and facial recognition. As with the blind-spot imaging system and the forward looking imaging system, the interior imaging system does not typically render or display the captured digital images; rather, digital images captured with the interior-view camera 112 are analyzed by a signal processor integrated into and dedicated to the interior imaging system, and are deleted after analysis.
The rear-view imaging system is typically used for the applications of back-up assistance and parking assistance. Typically, digital images captured by the rear-view camera 113 are rendered and displayed on a video display to provide live video to the operator of the motor vehicle 105. The captured digital images may also analyzed by a signal processor integrated into and dedicated to the rear-view imaging system in order to provide warnings to the operator. The captured digital images are not stored, and are deleted after they are displayed or analyzed.
Prior art automotive imaging systems have been described for accident detection and subsequent video image capture and recording. Two examples of such systems are U.S. Pat. No. 6,141,611 to Mackey et al., entitled “Mobile vehicle accident data system,” and U.S. Pat. No. 7,348,895 to Lagassey, entitled “Advanced automobile accident detection, data recordation and reporting system.” In U.S. Pat. No. 6,141,611 a system is described where one or more video cameras are mounted on a vehicle to continuously record the vehicle surroundings. This video data is stored on-board along with other operational parameters and data. The on-board storage memory is a First-In-First-Out (FIFO) buffer with a specified size, so that only the most recent images are available, and earlier images are overwritten. Upon detection of an accident, the stored images and data are locked and automatically transmitted wirelessly to a central database. Continuous recording of all cameras requires a very large memory and captures information that is not relevant to the accident at the same rate and resolution as image captured during the accident. No images are captured after the accident. Additionally, no provisions are made for multi-use of the one or more cameras mounted to the exterior of the vehicle or for flexible operation of the accident recording system. This fixed and dedicated function is not cost effective. This is pointed out by U.S. Pat. No. 7,348,895 which proposes accident detection and imaging systems at intersections rather than in a motor vehicle. While this solves the problem of high cost, it does not provide adequate coverage for all accidents since many accidents do not occur at intersections.
Other prior art automotive imaging systems are directed at security and anti-theft. Two such instances are described in U.S. Pat. No. 7,500,794 to Clark, entitled “Concealed vehicular camera surveillance system,” and U.S. Patent Application Publication 2003/0081121 to Kirmuss, entitled “Mobile digital video monitoring with pre-event recording.” These references are directed to imaging systems located in the motor vehicle of a law enforcement authority to record traffic stops, or surveillance of other suspected crimes. Both are dedicated imaging systems and are not integrated with other systems and do not have multiple functions. Both require activation and operation by the operator or law enforcement officer.
In U.S. Patent Application Publication 2008/0061951 to Liu, entitled “Image transferrable anti-theft system,” a motor vehicle anti-theft system is disclosed. This system utilizes a first camera in the cabin of the vehicle and an optional second camera the cargo area of the vehicle. The cameras are activated only when an entry door is forcibly opened. Upon subsequent starting of the vehicle, video images are transmitted to the vehicle owner's PDA, computer or phone to prompt the owner to save or store the images.
In U.S. Pat. No. 7,602,947 to Lemelson et al., entitled “Facial-recognition vehicle security system,” a facial-recognition vehicle security system is described. This system employs one or more interior mounted cameras to view the vehicle operator. If the face of the operator is not recognized as an authorized operator the vehicle is disabled. The video images of this event are not recorded or sent to a third party.
U.S. Patent Application Publication 2008/0165251 to O'Kere, entitled “Camera systems and methods for capturing images in motor vehicles,” also describes a camera system for monitoring the interior of a motor vehicle. Cameras are activated by an entrance door opening, by the starting of the vehicle, or manually by a push button in the trunk of the vehicle. In this reference, the images are stored, and a provision for live video transmission to law enforcement authorities or a monitoring service is discussed. In these references the camera systems are specialized or dedicated systems, and are limited to imaging of the vehicle interior and activated only by forced entry or attempt at unauthorized use.
U.S. Pat. No. 7,537,400 to Schmid, entitled “Camera system for a motor vehicle,” describes an automotive imaging system with universal camera mount locations located at the front, rear side and interior of the vehicle. These are connected to a central information processing unit in the vehicle. The connection can be wired or wireless. Each mount location is dedicated to a specific imaging function. For example, a camera placed in the rear mount is used only for heads up rear view imaging. In this system, the cameras are simply image capture devices and all image processing, image analysis, and subsequent action/decision based on the analysis for the various imaging functions is done in a central image processing unit. This has the disadvantage of moving all of the analysis and camera sub-system complexity to a central unit. The performance of the central unit must now be extremely high in order to perform all of the possible image analyses and subsequent decisions and information display in parallel. This would make the cost of the integrated system higher. Accident recording is stated as one of the functions that could be carried out by a forward looking camera.
U.S. Patent Application Publication 2003/0098909 to Fritzsche et al., entitled “Process for monitoring the internal space of a vehicle, as well as a vehicle with at least one camera within the vehicle cabin,” teaches using a panoramic camera to monitor the interior of a motor vehicle. Images captured by the camera can be transmitted to an emergency aid station in the case of an accident, or to the owner in the case of the activation of an alarm system.
U.S. Patent Application Publication 2007/0216136 to Dietz, entitled “Single camera apparatus and methods for alignment of a trailer hitch,” teaches the use of a camera to provide images to a driver to aid in the alignment of a trailer hitch. The system can be configured to transmit images from the camera to the owner in the case of an alarm system activation.
U.S. Pat. No. 7,423,529 to Singer et al., entitled “Systems and methods for mobile security and monitoring,” a surveillance system for use in a mobile environment. In some cases, the system utilizes a variety of image and audio sensors to record activity ongoing in the mobile environment. In some embodiments, the recorded activity can be transmitted wirelessly to a remote location.